Summary

Adds a new provider: localbm25, an offline BM25 baseline memory provider for MemoryBench.

Motivation

MemoryBench currently requires a hosted memory provider (Supermemory/Mem0/Zep) to run ingestion + retrieval. This introduces friction for contributors who don’t have provider credits.
localbm25 provides a fully offline baseline provider (in-memory BM25) so anyone can run MemoryBench without provider API keys and establish a reproducible baseline.

What’s included

• New provider implementation: src/providers/localbm25/
• Provider registration + config wiring (ProviderName, provider registry, getProviderConfig)
• TypeScript module declarations for untyped wink packages
• CLI help + docs updated to include localbm25
• Dependency updates (wink-bm25-text-search, wink-nlp-utils)

Usage

Run MemoryBench with the offline baseline provider:

bun run src/index.ts run -p localbm25 -b convomem -j gpt-4o -r local_run --limit 50 --force

Compare it against hosted providers:

bun run src/index.ts compare -p localbm25,supermemory,mem0,zep -b convomem -s 5 -m gpt-4o-mini -j gpt-4o

✅ Note: localbm25 does not require any provider API keys (only judge/answer model keys).

Implementation notes

• Sessions are ingested as stringified JSON and indexed using BM25 (wink-bm25-text-search).
• Index consolidation happens once after ingest (awaitIndexing) to match MemoryBench’s ingest → indexing → search pipeline.
• For very small doc collections (e.g. --limit 1 / --limit 2), BM25 consolidation/search may not be supported by wink; provider falls back to deterministic token-overlap retrieval to keep the pipeline functional.

Validation

• convomem run completed end-to-end with localbm25 and generated data/runs/<run-id>/report.json.
• compare successfully initializes and runs ingestion/indexing/search for localbm25 (answer/eval model rate limits depend on configured judge/answer model and org limits).

Why this is useful

• Enables a reproducible offline baseline for retrieval and memory evaluation
• Lowers contribution barrier (no hosted provider credits required)
• Provides a consistent baseline to compare hosted memory providers against